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We at the National Network of State Teachers of the Year (NNSTOY 
are most pleased to share with you the latest in our series of Research 
Reports. 


In this report, we focus on measures of social and emotional learning 
and their efficacy in educator evaluation. Using multiple approaches 

— literature review and three focus group panels — we examined what 
the field says about these measures and well as what expert teachers — 
State and National Teachers of the Year and Finalists for State Teacher 
of the Year think. 


| worked for twelve years at Educational Testing Service in the area of 
performance based assessment across a continuum of educator practice. One of the first maxims 
that | learned was: ’...not everything that is important can be measured and not everything that 
can be measured is important.’ A second was '...we must not use assessment measures for pur- 
poses for which they have not been developed or validated.’ 


| also had the privilege of working with a team of expert research scientists, including Dr. Richard 
Roberts, to examine the efficacy of assessing social and emotional characteristics and their mallea- 
bility in educators. So, this study is of particular interest to me. 


Can we actually measure social and emotional characteristics, and, are they malleable? In other 
words, can we teach things like empathy, caring, persistence? Should we include measures of 
these characteristics in teachers as part of consequential evaluation? As expert educators, we 
deeply desire accountability measures. We want those measures to be grounded in research and 
to be valid, reliable, and fair. 


Working with our study partners, EducationCounsel on the policy side and Research Triangle Insti- 
tute on the science end, we are eager to share our results. We found the following key points: 


¢ Current research shows that social emotional skills can help students achieve successful school outcomes 
and that teachers can play a fairly sizable role in developing students’ social emotional skills. 


e Teachers in our focus groups agreed with much of the current research that social emotional development 
is critical and teachers and schools, through classroom-based and extracurricular activities, do contribute 
to students’ social emotional development. 


e Teachers in our focus groups agreed that professional development around social emotional development 
would be helpful and states should place focus there, but many questioned the feasibility of doing so given 
the current emphasis on cognitive development (i.e. test score performance). 


e Teachers in our focus groups questioned whether they currently had sufficient resources or supports to 
devote time to social emotional learning. 


e Teachers in our focus groups thought that reports about students’ social emotional learning could be help- 


ful, but they were highly skeptical of using social emotional development in teacher evaluation systems. 


At NNSTOY, we believe that educators should always be at the table when education policy is 
being crafted, debated, or modified. As professionals, we know the most about what is likely to 
directly impact students and the work in the classroom, both positively and negatively. We are 
excited to share this paper with you and look forward to working with you in bringing the voice of 
educators to the policy process. 


With warm regards, 


Vothesies: Bowesst 


Katherine Bassett, Chief Executive Officer, NNSTOY 


NNSTOY 


TEACHERSLEADING 


“PRACTI 


Acknowledgements 


NNSTOY wishes to thank the following individuals and groups for their support and contributions 
to this project: 


Our Partners 

Our partner in this work, EducationCounsel, specifically Ms. Bethany Little, for her commitment to 
learning what a group of outstanding educators thought about this important topic. Their guid- 
ance, policy expertise, and assistance with the work overall was invaluable, as was their overall 
collaboration. 


Our science partner in this work, Research Triangle Institute (RTI). Dr. Elizabeth Glennie, Dr. Jeffrey 
Rosen, and Ms. Lisa McCaskill were outstanding partners, providing scientific expertise and a com- 
mitment to working side-by-side with NNSTOY teacher researchers. 


We had two teacher researchers contributing their talents to this study. Dr. Rebecca Snyder and Dr. 
Maryann Woods-Murphy are gifted educators, whose pursuit of continuous learning has led them 
to earning their advanced degrees. Their thoughtfulness and expertise from the classroom, added 
to their scientific experience, made them outstanding contributors to this work. 


We thank both sets of partners for their patience, dedication, and collaboration in this lengthy 
process. 


Our Funders 

We were fortunate to have generous funding with which to conduct this study supplied by the 
Raikes Foundation. Their sincere interest in this topic is inspiring. Without this funding, this study 
would not have taken place. We are most grateful. 


Our Reviewers 

We sincerely thank the following for making the time to conduct an external review of this report: 
Dr. Hunter Gehlbach of the University of California, Santa Barbara; and Dr. Laura Goe of Educa- 
tional Testing Service gave graciously of their time and expertise to strengthening this work. 


The Focus Group Participants 

Finally, we could not have asked for a more prepared and committed set of educators with whom 
to do this work. Each focus group participant brought deep thoughtfulness about these issues as 
well as direct experience in working with measures of social emotional learning. Each gave gener- 
ously of their time in thinking seriously about the issues considered and we are extremely grateful 
to them. 


Table of Contents 


Student Social and Emotional Development and Accountability: 

Perspective of Teachers 
ool Lo: i ee ene ae eee eee em ee reer 
Methods 7 
Grit, Growth Mindset, Belonging and Student Success 
The role of teachers and schools in fostering social and emotional 


competencies and skills 9 


Focus Groups - Teacher's role in social and emotional development... 10 
The use of data about social and emotional competencies and skills 11 
Focus Groups - The use of data about social and emotional 

competencies and skills 12 


The use of measures of social and emotional development 


in accountability systems 
Focus Groups - Measures of social and emotional learning 
In accountability SyStEMS oo ommmmmnssnavnnuuane 


CONCIUSION iii 14 
Appendices 

REFEPENCES ei 16 

Appendix A: Focus Group Protocol cnn 18 


Appendix B: Characteristics of Focus Group Participants (N=28)........20 


The National Network of State Teachers of the Year | Student Social and Emotional Development and Accountability: Perspective of Teachers 5 


NNSTOY 


TEACHERSLEADING 


“PRACTI 


Introduction 


Researchers and practitioners are learning more and more about the importance of inter- and 
intra-personal (often called “social and emotional”) competencies and their impact on student suc- 
cess both in and out of school. Inter- and intra-personal competencies include attitudes and behav- 
iors that affect how students reflect on and apply their learning (intra-personal) and skills needed 
to relate to and manage relationships with others (inter-personal). These are known in the field by 
many different terms, including social and emotional, 21st century skills, deeper learning, non-ac- 
ademic, non-cognitive, and soft skills, that are overlapping frameworks of competencies and skills 
beyond academics that students need for success. For example, the Collaborative for Academic, 
Social and Emotional Learning (CASEL) identifies specific categories of social emotional competen- 
cies, including self-awareness, self-management, social awareness, and relationship management. 
Others in the field see a different, fundamental set of components that help students succeed, 
including connectedness, motivation, and self-regulation (Gelbach, 2015). 


Social and emotional competencies and skills help students succeed in school. In fact, researchers 
have documented significant academic gains for students who participate in social and emotional 
learning programs in which they can strengthen their social and emotional skills (Durlak, Weissberg, 
Dymnicki, Taylor, & Schellinger, 2011). Skills such as growth mindset (Blackwell, Trzesniewski, & 
Dweck, 2007), sense of belonging (Allen, Kern, & Vella-Broderick, 2016a; Allen, Vella-Broderick, 

& Walters, 2016b), and grit (Duckworth & Quinn, 2009) have been shown to correlate to student 
grades, scores on achievement tests, attendance and other academic outcomes. 


Given the influence that social and emotional skills have on student academic outcomes, some 

policymakers are considering using measures of these skills to assess the performance of teach- 
ers and schools. In some states and localities, student perceptions of their social and emotional 
development are being considered for high-stakes accountability purposes, with incentives and 
consequences for schools based upon scores on these measures. 


This paper uses both a systematic review of research on these competencies and skills as well as 
focus groups with National Network of State Teachers of the Year (NNSTOY) teachers to examine 
whether measures of social and emotional learning are appropriate for use in accountability sys- 
tems. Specifically, we consider the following topics: 


1. the role of teachers and schools in fostering social and emotional competencies and skills, 
2. the use of data about social and emotional competencies and skills, and 


3. perceptions about the appropriate use of measures of social and emotional development in 
accountability systems. 


For the first two questions, we examined the academic studies that have addressed it, and then 
discussed the questions with teachers in our focus groups. For the third question, we described 
policies for including these measures in accountability systems and researcher opinions about do- 
ing so, and then discussed the question with teacher in our focus group. 


2 Pellegrino J. W., Hilton M. L. (2012). Education for life and work: Developing transferable knowledge and skills in the 21st century. 
Washington, DC: National Academy of Sciences. 


Methods 


To permit in-depth investigation, we focused the review of research on three skills that have re- 
cently received a great deal of attention from researchers, policy-makers, and educators, as well as 
the media: grit, growth mindset, and sense of belonging. Grit is defined as the tendency to sustain 
interest in, passion for, and effort toward long-term goals (Duckworth, Peterson, Matthews, & Kelly, 
2007). Growth mindset is the extent to which students believe that their intelligence can improve 
with effort, rather than being fixed by factors outside of their control (Dweck, 2006). Sense of be- 
longing is most commonly defined as the extent to which students feel accepted, respected, val- 
ued, and included in their school environments (Allen et al., 2016a; Allen et al., 2016b; Goodenow 
& Grady, 1993). Grit, growth mindset, and sense of belonging are, at their core, a combination of 
social, motivational, and self-regulation skills. 


In the review of research, we searched for recent academic articles citing grit, growth mindset 

or sense of belonging in the Education Resources Information Center (ERIC), PsycINFO, EBSCO, 
ProQuest, and Google Scholar databases. Prominent education blogs and news outlets, such as 
Education Week, yielded general news articles or posts featuring leading researchers in these 
areas, such as Duckworth, Yeager, Walton, and Dweck. To investigate the way that states and 
localities are incorporating measures of these skills into accountability systems, we examined web 
sites of state departments of education, collected news articles and data from blogs that indicated 
state interest in social and emotional learning, and obtained research reports from educational 
councils and foundations, such as the National Council on Teaching Quality and the Great Schools 
Partnership. 


The review of research sought evidence of the importance of these skills for students’ success in 
school. To understand teachers’ perspectives on social and emotional learning and accountabil- 
ity, we conducted focus groups with 28 NNSTOY teachers during the week of January 30, 2017. 
Through the literature review, we identified states and districts in which data were being collected 
about student social and emotional learning and recruited NNSTOY teachers from those areas.° 
Each participant in the focus group had experience in using results from data about student social 
and emotional learning. There were three separate focus groups, each of which met via videocon- 
ference for about 1 hour and 15 minutes. 


By conducting focus groups with teachers, we obtained their views about how they believe they 
can influence these skills and how schools should use data about them. To capture the full range 
of teacher experiences with and their impressions of social and emotional learning, we did not 
limit the discussion to grit, growth mindset, and sense of belonging.. Teachers were invited to dis- 
cuss their strategies for developing any social emotional competencies and skills and the extent to 
which they and their schools used data to learn about students’ social and emotional development. 


Appendix A contains the focus group protocol. Before conducting the focus groups, we admin- 
istered a brief survey to participants asking them about the characteristics of their schools. Most 
of the participants were experienced teachers; only two had fewer than 11 years of experience. 
Half of the participants worked in suburban schools, while a quarter of them worked in urban 
schools, and a quarter worked in rural schools. About half of the participants taught at the second- 
ary school level, and fewer than 20 percent were in schools with more than 1,500 students. One 


3 We recruited NNSTOY teachers from the following states: Colorado, Connecticut, Delaware, Georgia, Hawaii, lowa, Illinois, Kentucky, 
Massachusetts, Maine, Michigan, North Carolina, New York, Rhode Island, Utah, and Washington. 
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quarter of them taught in schools where at least two thirds of the students were eligible for free or 
reduced-price lunch. Appendix B presents additional details on the characteristics of the sample. 


Grit, Growth Mindset, Sense of Belonging and Student Success 


Much research has examined the ways that grit, growth mindset, and sense of belonging can ben- 
efit students and the ways in which teachers and schools can influence these skills. One early study 
of grit’s relationship to academic achievement found that “grittier” adolescents had higher grade 
point averages (GPAs) (Duckworth & Quinn, 2009). More recent studies continue to show the 
positive impacts of grit on student outcomes including attendance, behavior, and test score gains 
between the 4th and 8th grades (West et al., 2016). 


However, some have questioned how much grit contributes to explaining academic performance 
independently of other social and emotional skills. A study of secondary school students in the 
United Kingdom found that grit contributed very little to grades on the General Certificate of Sec- 
ondary Education exams once analyses accounted for other characteristics, such as conscientious- 
ness. Additional results from this study suggested that teachers may not have much influence over 
improvement on this skill (Rimfield, Kovas, Dale, & Plomin, 2016). 


Similarly, a study of high school students in the U.S. showed that grit did not predict academic 
outcomes such as rule-violating behavior, academic recognitions, honors, and GPA, but conscien- 
tiousness and emotion regulation did (Ivcevic & Brackett, 2014). Grit did not add much value to 
understanding student performance when analyses accounted for other social emotional skills. A 
recent meta-analysis concluded that grit exhibited only very modest relationships with academic 
performance and those relationships did not compare favorably with other predictors of academic 
outcomes, such as study habits and skills (Credé, Tynan & Harms, 2016). 


Growth mindset, or a belief among students that they can grow their intellectual capacity, appears 
to be a strong predictor of academic achievement for elementary and secondary students. Key ex- 
perimental intervention studies have shown that a growth mindset predicts improvements in math 
over time. In one study, seventh-grade students whose math scores were in decline received week- 
ly information sessions on how the brain develops and grows with use. Students who received that 
information stabilized their declining math grades while those who did not continued their decline 
(Blackwell et al., 2007). Another study of math performance found that female and ethnic minority 
students who were encouraged to view intelligence as something that could be increased with 
additional mental work earned significantly higher scores on standardized math tests than study 
subjects not exposed to the intervention (Good, Aronson, & Inzlicht, 2003). Growth mindset was 
positively correlated with attendance, behavior, and math and reading test score gains between 
the 4th and 8th grades (West et al., 2016). A review of the research on resilience and academic 
performance reported that students who believed that intellectual abilities could be developed (as 
opposed to being fixed qualities) tended to show higher achievement across challenging school 
transitions (e.g., the transition from middle to high school) and greater course completion rates in 
advanced math courses (Yeager & Dweck, 2012). 


Sense of belonging in school refers to the extent to which students feel accepted and valued in 
school. Some research has found positive effects of school belonging on important academic out- 
comes. For example, a greater sense of belonging in school has been shown to relate to improved 


grades and academic competencies (Pittman & Richmond, 2007), higher levels of academic mo- 
tivation (Gillen-O'Neel & Fuligni, 2013), and lower levels of negative academic behaviors such as 
absenteeism, dropping out, truancy, and academic misconduct (Demanet & Van Houtte, 2012). In 
general, research shows that students who feel a greater sense of belonging perform better on a 
range of academic performance measures (Wang & Holcombe, 2010), and that a sense of belong- 
ing may help some students more than others. Members of historically disadvantaged groups in 
math or science programs, such as ethnic minorities and females, may worry more about belong- 
ing, thus increasing their academic stress, which can ultimately lead to poorer academic outcomes 
(Yeager, Walton & Cohen, 2013). 


The role of teachers and schools in fostering social and emotional 
competencies and skills 


If social and emotional skills are malleable, then they should be influenced by interventions such as 
teaching. A growing body of research examines the influence that teachers can have on social and 
emotional skills in students. Studies have had mixed results. For grit, some researchers continue to 
encourage further research examining its malleability (Duckworth & Gross, 2014), but others have 
questioned both its importance for academic achievement (Credé et al., 2016) and its malleability 
(Rimfield et al., 2016). For growth mindset, there is a solid research base suggesting that it can 

be changed, even by relatively small interventions. A review of interventions targeting students’ 
beliefs about their potential for growth and sense of belonging in school showed that students re- 
ceiving these interventions improved on outcomes like math achievement and overall GPA (Yeager 
& Walton, 2011). Sense of belonging in school appears to be influenced by many factors that occur 
outside the school setting, such as emotional stability, parental support, and peer support. The 
in-school factors include teacher support and environmental/school safety. Importantly, elements 
of teaching have been shown to be among the stronger influences on sense of belonging in school 
(Allen et al., 2016a), but the sense of belonging develops due to factors both within and outside 
the control of teachers and schools. 


Consistently, researchers have found that teachers are the most important in-school factor in pro- 
ducing improved student achievement (Kane & Staiger, 2012). Beyond direct instruction, teachers 
can influence the social and emotional development of their students in many ways. For example, 
when students perceive a caring, enthusiastic, supportive, and available teacher, their sense of be- 
longing in school improves (Allen et al., 2016a; Wang & Holcombe, 2010; Demanet & Van Houtte, 
2012). When teachers challenge students to think more rigorously by striving to understand 
concepts and to explain their reasoning, students’ growth mindset increases (Ferguson, Phillips, 
Rowley, & Friedlander, 2015). Teachers can have significant influence over student behaviors, such 
as absences, that tend to affect academic outcomes (Ladd & Sorenson, 2014). 


A growing body of research is providing estimates on how much influence teachers have on social 
and emotional development. Research has found substantial variation in teacher contributions to 
students’ grit, growth mindset, and effort suggesting that some teachers are more effective than 
others in improving these skills in students (Kraft & Grace, 2016). A study examining the extent to 
which teachers influenced students’ approaches to learning, interpersonal skills, and self-control 
found that kindergarten teachers had larger effects on their students’ behavioral skills than on 
their academic skills (Jennings & DiPrete, 2010). Upper-elementary teachers had large effects on 
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self-reported measures of students’ self-efficacy in math, and on classroom happiness and behav- 


ior. Teachers described by students as emotionally supportive tended to have students with higher 
math self-efficacy and better classroom behaviors (Blazar & Kraft, 2016). 


Focus groups — teacher's role in social and emotional development 


In focus groups, teachers discussed the role of teachers and schools in developing students’ social 
and emotional skills. Participants identified many ways in which they and the school setting could 
help students cultivate these skills. Many teachers in the focus groups said that within the class- 
room, practices such as cooperative learning and coaching can help students succeed in school. 
School-wide character education programs can create opportunities to embed social emotional 
learning in the classroom. One teacher described a program at her school. “We focus on building 
positive relationships with students. At the high school level, they have student-voice teams. They 
deal with different platforms and speak from that students’ 
platform on issues taking place in school and classrooms.” Two 
mA AV(<Ma KoLetU [Moya ol0li(ol late! respondents mentioned that permitting students to re-do 
positive relationships with assignments, quizzes, or tests could help them develop per- 
a0 (o(-\ala-Mw- Nam dal-Mallo] alot alofo) sistence in their learning. A few teachers observed that flexibil- 
level, they have student- ity in the school schedule would help, because students could 
voice teams. They deal with meet in small groups with teachers or counselors, and students 
elhic-le-laias o){-lakelgaatw-lare| could connect with each other. 

speak from that students’ 
platform on issues taking 
fe) F-Xer- mia e-xeuatete) = lave, 
classrooms.” 


Many focus group participants noted that extracurricular activ- 
ities provide another mechanism for schools to foster students’ 
social and emotional skills. In these activities, students develop 
supportive relationships with other students, teachers, and 
other adults. A few respondents believed that service proj- 
ects help students work together to develop these skills, and 
one teacher mentioned Girls on the Run as a positive program. Another believed that before- or 


after-school wellness programs help. Some teachers mentioned helpful connections outside the 
school and described community outreach through volunteers, social workers, or community pro- 
grams, such as the Second Step program. 


Ultimately, these teachers believed that by developing strong relationships with students, they can 
help them strengthen social and emotional skills. Numerous teachers thought that teachers need 
to model effective social and emotional practices themselves. When teachers demonstrate these 
skills, students can learn by interacting with teachers. Student-teacher relationships provide a foun- 
dation for helping students develop social and emotional skills. 


Many teachers in these focus groups described how they had Ultimately, these teachers 


believed that by developing 
strong relationships with 


taken the initiative to cultivate student relationships, through 
activities such as having lunch with students, asking about their 
needs, and spending a little of bit of class time having students 
share their learning goals and needs. 


students, they can help 
ival-Janmi4e-lave id al-lamxexe[-1 il ale) 
However, teachers identified challenges within schools to help- emotional skills. 

ing students develop social and emotional skills. Many teachers 


mentioned that policy emphasis on academic exams means that teachers do not have time to 
spend on their students’ social emotional learning. Some teachers noted that they do not con- 
trol their schedules and cannot choose to set aside time for activities that might foster students’ 
social and emotional learning. One high school teacher said it was risky to take class time away 
from academic content to address social emotional learning, and others mentioned challenges in 
the school schedule. Teachers must work with federal, local, or state initiatives that they did not 
choose or develop. Responding to many new initiatives, which may have competing demands, can 
lead to a feeling of “initiative overload.” 


Many respondents raised concerns about whether teachers have resources and supports to help 
students with social and emotional learning. Some focus group participants noted that some stu- 
dents may need more help than a teacher can or should provide, such as someone who needs the 
help of a mental health professional. Teachers need to know where they can get help in these cir- 
cumstances. Even in circumstances that are within the teacher's direct influence, a few focus group 
participants said that it is important to be sensitive to students’ specific needs. Students living 

in poverty, of various racial/ethnic groups, with disabilities, or with a variety of family challenges 
may have differing social emotional learning needs. Teachers may not know how to relate to the 
challenges some students face. Some teachers may be afraid of making a difficult situation worse 
by saying or doing the wrong thing. 


A few mentioned that professional development in this area could help strengthen their ability to 
help students develop social and emotional skills. However, one commented that the professional 
development she received on social and emotional learning treated it as a standalone unit and did 
not highlight ways to integrate social and emotional learning into regular classroom activities. Two 
focus group participants mentioned that some teachers may think their job should focus exclusive- 
ly on students mastering content, and that they have not been trained to help students develop 
these skills. Another noted that when social emotional learning is offered as one option among 
many workshops or seminars, not all teachers choose to participate in this strand of the profession- 
al development. 


The use of data about social and emotional competencies and skills 


Given that teachers may be able to foster students’ social emotional development, which can then 
impact students’ academic performance, some states and districts are collecting data about mea- 
sures of student's social emotional skills. School systems get information about these attributes 
through the local use of student self-report surveys that assess student perceptions of their own 
development, their teachers, and the school climate. A widely-used student survey called the Tri- 
pod has been administered in school districts such as Pittsburgh, Memphis, and Denver to provide 
teachers both diagnostic and professional development information. According to the developers 
of Tripod (see http://tripoded.com/teachers/), teachers can gain insights from student perceptions 
of instructional practice, including a better understanding of what students are experiencing, iden- 
tification of areas of strength and opportunities to improve, and ways to increase student engage- 
ment in academic activities. This information can assist teachers in tracking their own progress over 
time. 


Based on evidence linking student perceptions of teaching practice to improved academic out- 
comes (Kane & Staiger, 2012), some states and districts are taking steps to incorporate student 
surveys into accountability systems. For example, Alabama includes student surveys in its teacher 


4 The CORE districts are the 8 largest districts in California: Fresno, Garden Grove, Long Beach, Los Angeles, Oakland, Sacramento, 
San Francisco, and Santa Ana Unified. 
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evaluation process, basing 10 percent of teacher evaluation scores on these surveys. In Connecticut, 
student survey results account for 5 percent of a teacher's evaluation score. Other states, like Hawaii 
and New York, have made student surveys an important part of teacher rating systems. In terms of 
using measures of social-emotional growth in school accountability systems, the California Office to 
Reform Education (CORE) districts of California* are further along than most. These districts are cur- 
rently allotting 8 percent of school accountability ratings to social emotional learning measures, as well 
as another 8 percent to climate and culture surveys of parents, teachers, and school staff. 


Focus groups -— the use of data about social and emotional competencies 
and skills 


In the focus groups, teachers discussed using data about students’ social and emotional learning. 
Everyone who participated in the focus groups had seen results from data about student social and 
emotional learning. In most cases, data were collected from students, but some teachers noted that 
information was also collected from teachers, school staff, and parents. Some of the surveys were 
designed to assess students’ social and emotional skills. Two teachers mentioned the Tripod survey, 
another cited a survey about growth mindset, and a fourth described a survey about connectedness, 
or sense of school belonging. Most teachers described surveys that do not measure students’ social 
and emotional skills, but instead measure assets needed to build those skills, such as students’ rela- 
tionships with teachers, parents, and other trusted adults. Two teachers in the focus groups reported 
administering their own student surveys so that they could understand how best to respond to and 
support their students. 


For the most part, focus group respondents who were given reports about student social and emo- 
tional learning received aggregate data about their schools or classrooms. Students, as well as teach- 
ers, sometimes received aggregate results about the school climate. In some cases, results were used 
for professional development activities and goal setting, or to connect students with community 
services. 


The use of measures of social and emotional development in accountability 
systems 


Some research has suggested that social and emotional measures might be included in accountability 
systems, and researchers and policymakers debate whether doing so is appropriate. A recent analysis 
of social emotional skills in the California CORE Districts noted a strong correlation between students’ 
average social emotional skill ratings and their schools’ scores on math and reading examinations, as 
well as student grade point averages. Lower ratings were correlated with higher school suspensions 
and absenteeism in middle school (West et al., 2016). Though only correlational, these results have en- 
couraged the California CORE districts to move forward with including social emotional learning mea- 
sures in school accountability reporting. In the coming years, the CORE districts will include growth 
mindset, self-efficacy, self-management, and social awareness measures in school-level reporting.® 


While the CORE districts and others move forward, serious concerns have been raised about methods 
used to measure these attributes. Currently, social and emotional learning attributes are measured pri- 
marily using self-reports. Duckworth and Yeager (2015) are cautious about the use of such approach- 


5 http://coredistricts.org/our-work/social-emotional-learning/. 


es for several reasons. For example, measurements of social 
Even proponents of the use emotional skills may suffer from reference bias when a student 
of student surveys have responds based on comparisons with others, rather than on an 
urged caution. For example, objective measure. Survey respondents may also provide the 
iNodat-1(oll m-lee [Xelalr-lave mel dat-1g answer they believe is socially desirable rather than an honest 
fol -WW/-\ fo) of -1e-Mo) i wa—ma a] oxere: answer. 


survey encourage its use 
as a tool for professional 
learning, but not as a 
punitive measure. 


Even proponents of the use of student surveys have urged 
caution. For example, Ronald Ferguson and other developers 
of the Tripod survey encourage its use as a tool for professional 
learning, but not as a punitive measure: 


Tripod has encouraged the use of surveys to promote profes- 
sional learning. Tripod can enhance teacher evaluation systems 
by providing an additional perspective, but we always want the major focus to remain instructional 
improvement. 

(http://tripoded.com/districts-states) 


Focus groups — measures of social and emotional learning in accountability 
systems 


Teachers in the focus groups did not mention being evaluated based on their students’ social and 
emotional learning, although some knew of plans for their states or districts to do so. Some focus 
group members thought that if measures of student social and emotional skills were included in 
a school accountability system, such 
results could put a school’s academic 


test scores in context, and could iden- Many teachers described the importance of 

tify the kinds of professional develop- itll Cola lave M-kS-1-1¢-]aleoM kom] 40 [e(-Jalecm ey-\-To mela tal-yie 
ment and community resources that circumstances and challenges and expressed 
would assist schools in helping their concern that putting this information into an 
students. However, most participants accountability system would make it a formulaic 
expressed concern about the pros- one-size-fits-all approach. 


pect of including information about 
students’ social and emotional learn- 
ing in an accountability system. Many 
teachers described the importance of tailoring assistance to students based on their circumstances 
and challenges and expressed concern that putting this information into an accountability system 
would make it a formulaic one-size-fits-all approach. This assistance needs to be sensitive to cul- 
tural differences among students. Teachers in the focus group wanted to act on information about 
students’ social and emotional learning and were concerned that within an accountability system, 
they would not get data about their students until the end of the year, when it would be too late 
to help them. 


Some teachers in focus groups questioned the validity of the measures of social and emotional 
learning. One noted that the findings about topics such as grit and growth mindset change over 
time. A state accountability system could not respond quickly to changes in what is known about 
these factors. Other teachers noted that students may not complete surveys accurately, particu- 
larly if they know that results will be used in the accountability system. If students do not trust the 
people collecting the data, they will not share openly. If schools do not collect data from students, 
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and instead use administrator classroom observations to measure student social emotional skills, 


administrators will not be able to capture much information in a short time. 


No one in the focus groups thought that measurements of these skills should be used in teacher 
evaluations. Many teachers believed that incorporating social and emotional skills in an account- 
ability system would hurt morale, particularly in high-needs schools. Because a high percentage of 
students in high-needs schools may be facing personal and family challenges, such as food insecu- 
rity, those students may have lower average scores than those in more advantaged schools, so this 
approach could be harmful to high-needs schools. It could also work against high-needs schools if 
community members who see these 


results think the school’s problems are 
...it is not clear which skills have the greatest insurmountable. 
influence on students’ success, which skills are 


sd at-Wantey-ian ants} i(-¥s]e)(-Wmelandal(ealauxsiavc-)alute)atcirela=) Conclusion 


the most effective. ae 
Social and emotional learning includes 


a range of competencies, such as 
self-regulation, resilience, grit, mind- 
sets, efficacy, and belonging. The materials we reviewed suggest that many social and emotion- 
al skills have important benefits to students, but pose challenges for data collection. Research 


suggests that social emotional skills can help students succeed in school, and that teachers and 
schools have a significant role to play in creating an environment that fosters the development of 
these competencies and skills. However, it is not clear which skills have the greatest influence on 
students’ success, which skills are the most malleable, or which interventions are the most effective. 
In focus groups with NNSTOY teachers, participants shared 


their experiences and identified ways that schools and teachers _..research has not shown 
can help students. They also identified some of the challenges i oshvaaatelelalcvate Rll] alehe 

to fostering social and emotional skills for students of different types of preparation and 
ages from diverse backgrounds. Teachers in our focus groups professional learning 
thought that some kinds of professional development would teachers need to accurately 
help teachers, but research has not shown how much and what assess students’ needs and 
types of preparation and professional learning teachers need implement strategies to 

to accurately assess students’ needs and implement strategies Prolelasctomdalcvaal 


to address them. Participants believed that it is important to 
help students develop social and emotional skills, but that such 
help needs to be tailored to the needs of students. A uniform 
approach to working with students on social emotional learning will not be effective if it does not 
respect students’ needs. 


In addition to focusing on evaluating teachers and schools on students’ progress on social emo- 
tional learning, states should consider promoting professional growth opportunities for teachers 
in identifying and addressing students’ social and emotional learning needs. Scholars have devel- 
oped surveys that can give teachers diagnostic and professional development information. Some 
states and districts are using student surveys to collect data about students’ social and emotional 
skills. Teachers in the focus groups thought that results from data collection and analysis could 
inform professional development decisions at the school and identify community resources that 
could help students. 


Currently, using this information in an accountability system or for teacher evaluations is relatively 
rare. Some researchers who have pioneered the use of these concepts (Duckworth) and the instru- 
ments to measure social and emotional skills (Ferguson) have expressed caution about using these 
measures in this way. These tools were not designed for an accountability system, and student 
responses may not be valid or reliable in this setting. Teachers in our focus groups raised several 
concerns about using such information in an accountability system. They said that using a formula- 
ic approach to collecting and reporting data would not be an effective way to help students with 
their specific needs. If teachers do not get information in a timely way, they cannot help their stu- 
dents. One teacher said, “Anytime this data can affect how | work with or connect with students, 
then it is useful. But when it is standardized for the sake of collecting data and we won't get data 
back for months, then it is not useful.” They noted that bias can occur with self-reporting, and that 
students may not answer questions honestly if they know their 


answers will be used in an accountability system. 
These tools were not 


Using measures of these skills in an accountability system may ro l-sife ]al=Yo Mire) ar-lalr-exerole la) 
be detrimental. Many members of the community will not know ability system, and student 
how to understand the results, and may think that a school’s responses may not be valid 
problems are insurmountable. Further, although schools and or reliable in this setting. 
teachers can help students develop these skills, much of stu- 


dent development is beyond the control of schools and teach- 

ers, as families and communities influence social and emotional 

learning. In general, the scores of schools with high-risk populations will differ from those of more 
advantaged schools. Using such information in an accountability system may particularly penalize 
teachers in high-needs settings. The teachers in our focus groups did believe that schools and 
teachers can influence the development of students’ social and emotional learning and that they 
could learn from data about their students’ skills. However, such information must be used in way 
that will ensure that schools and teachers get the resources needed to help students. 
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Appendix A: Focus Group Protocol 


Consent 
Thank you for agreeing to be part of this focus group with NNSTOY and RTI International, 
sponsored by the Raikes Foundation. We appreciate your willingness to participate. 


The purpose of this group is to try and understand how your school collects data about students’ 
social and emotional skills, and how administrators and teachers use that data. We want to learn 
your perspectives about these important factors facing teachers and students today. The informa- 
tion learned will be collected in a report, which will be shared on NNSTOY's website. 


Your responses are critical to the success of this effort. Your participation is voluntary and you have 
the right to refuse to answer any question. Your decision to participate or not to participate will 
have no impact on your organization's relationship with NNSTOY. The purpose of this group is not 
to evaluate your particular school and your name will not be used in any reports. 


Although the focus group will be recorded, your responses will remain anonymous. 


Ground Rules 

There are no right or wrong answers. 

It's important to speak one at a time and to state your first name each time before speaking. 
It's important that everyone participate. 

It's okay to have differing points of view. 


1. Please introduce yourselves. Tell us your name, the state in which you teach, the grades and 
subjects taught, and your years of experience teaching. 
Before the call we shared a summary about nonacademic, or social and emotional, student 
skills - grit, sense of belonging, and growth mindset. Research shows these skills help students 
succeed in school. Some schools, districts, and states are collecting information about 
students’ social and emotional skills through activities such as student surveys. 


2. Does your school try to collect this type of information? 
[If so] How does your school do this? How do they collect data? How often do they collect 
data? 


3. Do you get the results of this information? 
[If so] How do you use the information from it? [Probe ... Do you get data about individual 
students or classroom-level data? Do you have opportunities for professional development 
opportunities aligned with the results? Does your school work with other agencies in the 
district to learn why students face challenges with nonacademic skills?] 


4. What aspects of the school environment do you think might help students develop these 
nonacademic skills? [Probe — culture, policies) 


5. In what ways do you think teachers can develop their students’ nonacademic skills? 


6. What challenges / obstacles do schools and teachers face in helping students develop these 
skills? 
Some schools, districts, and states are planning to use information they collect about student 
social and emotional skills in their accountability systems. 


7. ls your state/district doing / planning to do this? 


8. Are there advantages to incorporating information about nonacademic skills into an 
accountability system? 


9. Are there challenges to doing this? 


10. What kinds of information about helping students develop their nonacademic skills would be 
helpful to you? 
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Appendix B: Characteristics of Focus Group 
Participants (N=28) 


Table B1. Participants’ Years of Teaching Experience 


Number of Percentage (Number) 
Years Teaching | of Participants 

<=5 0 

6-10 7% (2) 

11-15 21% (6) 

16+ 72% (20) 


Table B2. Participants with Schools in each Locale 


Percentage (Number) 


Locale of Participants 
Rural 25% (7) 
Suburban itis 
tieah 25% (7) 


Table B3. Participants by Grade Level of Students Taught 


Percentage (Number) 


Grade level nee 
of Participants 


Elementary 29% (8) 
Secondary 54% (15) 
Other* 17% (5) 


*Includes participants who are no longer teaching in individual classroom, but rather working with students and/or multiple 
teachers as instructional coaches, through higher education teacher preparation programs, or through state education 


agencies. 


20 


Table B4. Participants by School Size 


Student Percentage (Number) 
exe) Ul E-haey a] of Participants 

< = 500 25% (7) 

501-1,000 43% (12) 

1,001-1,500 14% (4) 

1,501-2,000 4% (1) 

2,001+ 4% (1) 

Other* 10% (3) 


*Participants serve district- or school-wide or at the 
state level 


Table B5. Participants by School Percentage of Students Eligible for Free or Reduced-price Lunch 


Percentage of Students Receiving | Percentage (Number) 


al a-Y-Weolam nX-Yo [Ulek-Yo ml tale MUL ateta) of Participants 
< 33% 29% (8) 
33-66% 39% (11) 
67% + 25% (7) 
Other* 7% (2) 


*Participants are affiliated with or serve higher education or state education 
agencies. 
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